System and method for distortion characterization in fingerprint and palm-print image sequences and using this distortion as a behavioral biometrics

ABSTRACT

This invention uses a novel biometrics, called resultant fingerprints and palm-prints, for authentication. The novel biometrics are consecutive traditional print images where the subject physically changes the appearance of the print images by rotating or rotating and translating, or rotating, translating, and shearing the finger or palm. That is, it is a sequence of finger or palm-print images over a short interval of time where the images are modified according to the rotation or a combination of rotation and translation or a combination of rotation, translation, and shear. The rotational and translational and shear components of the motion in the sequence of print images are determined from the image-to-image flow. This flow is either computed from motion-compensation vectors of the sequence compressed in MPEG formats or directly from the uncompressed images. The global image-to-image flow is expressed in terms of an affine transformation, computed from the local flow in blocks around a non-moving central region. The rotational and translational components of this affine transformation are smoothed over a temporal neighborhood resulting in a function of time. This function of time is a behavioral biometrics which can be changed by the user when compromised. Matching of this function for authentication purposes is achieved very much as is done in legacy signature matching authentication systems where two temporal signals are compared.

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] The application is a continuation of U.S. patent application Ser.No. 09/537,077, filed on Mar. 28, 2000, which claims priority fromProvisional Application Serial No. 60/168,540, was filed on Dec. 2,1999.

FIELD OF THE INVENTION

[0002] This invention relates to the field of biometrics, i.e., physicalor behavioral characteristics of a subject that more or less uniquelyrelate to the subject's identity. This invention relates to a new typeof biometrics which is produced by a subject as a sequence offingerprint or palm-print images distorted through a series ofcontrolled changes to the traditional biometrics, by the motion of thefinger or palm over the print reader.

BACKGROUND OF THE INVENTION

[0003] Fingerprints have been used for identifying persons in asemiautomatic fashion for at least fifty years for law enforcementpurposes and have been used for several decades in automaticauthentication applications for access control. Signature recognitionfor authenticating a person's identity has been used at least forfifteen years, mainly for banking applications. In an automaticfingerprint or signature identification system, the first stage is thesignal acquisition stage where a subject's fingerprint or signature issensed. There are several techniques to acquire a fingerprint includingscanning an inked fingerprint and inkless methods using optical,capacitative or other semiconductor-based sensing mechanisms. Theacquired biometric signal is processed and matched against a storedtemplate. The image processing techniques typically locate ridges andvalleys in the fingerprint and derive the templates from the ridge andvalley pattern of a fingerprint image.

[0004] Signatures, on the other hand, are typically sensed through theuse of pressure sensitive writing pads or with electromagnetic writingrecording devices. More advanced systems use special pens that computethe pen's velocity and acceleration. The recorded signal can be simply alist of (x, y) coordinates, in the case of static signature recognition,or can be a function of time (x(t), y(t)) for dynamic signaturerecognition. The template representing a signature is more directlyrelated to the acquired signal than a fingerprint template is. Anexample is a representation of a signature in terms of a set of strokesbetween extremes, where for each stroke the acceleration is encoded. Forexamples of signature authentication, see V. S. Nalwa, “Automaticon-line signature verification,” Proceedings of IEEE, pp. 215-239,February 1997. This reference is incorporated herein by reference in itsentirety.

[0005] Recently, biometrics, such as fingerprints, signature, face, andvoice are being used increasingly for authenticating a user's identity,for example, for access to medical dossiers, ATM access, access toInternet services and other such applications.

[0006] With the rapid growth of the Internet, many new e-commerce ande-business applications are being developed and deployed. For example,retail purchasing and travel reservations over the Web using a creditcard are very common commercial applications. Today, users arerecognized with a userID and password, for identification andauthentication, respectively. Very soon, more secure and more convenientmethods for authentication and possibly identification involvingbiometrics, such as fingerprints, signatures, voice prints, iris imagesand face images will be replacing these simple methods ofidentification. An automated biometrics system involves acquisition of asignal from the user that more or less uniquely identifies the user. Forexample, for fingerprint-based authentication a user's fingerprint needsto be scanned and some representation needs to be computed and stored.Authentication is then achieved by comparing the representationextracted from the user's fingerprint image acquired at the time oflogon with a stored representation extracted from an image acquired atthe time of enrollment. In a speaker verification system a user's speechsignal is recorded and some representations is computed and stored.Authentication is then achieved by comparing the representationextracted from a speech signal recorded at logon time with the storedrepresentation. Similarly, for signature verification, a template isextracted from the digitized signature and compared to previouslycomputed templates.

[0007] Biometrics are distinguished into two broad groups: behavioraland physiological biometrics. Physiological biometrics, are the onesthat are relatively constant over time, such as, fingerprint and iris.Behavioral biometrics, on the other hand, are subject to possiblygradual change over time and/or more abrupt changes in short periods oftime. Examples of these biometrics are signature, voice and face. (Faceis often regarded to be a physiological biometrics since the basicfeatures cannot be changed that easily; however, haircuts, beard growthand facial expressions do change the global appearance of a face). Thefield of the present invention relates to physiological and behavioralbiometrics. More particularly, this invention relates to creatingresultant fingerprint and resultant palm-print biometrics by adding aseries of user-controlled changes to physiological or behavioralbiometrics and measuring the behavioral component. The representation ofsuch a resultant print is a template of the static print plus arepresentation of the user-induced changes.

[0008] One of the main advantages of Internet-based business solutionsis that they are accessible from remote, unattended locations includingusers' homes. Hence, the biometrics signal has to be acquired from aremote user in a unsupervised manner. That is, a fingerprint or apalm-print reader, a signature digitizer or a camera for acquiring faceor iris images is attached to the user's home computer. This, of course,opens up the possibility of fraudulent unauthorized system accessattempts. Maliciously intended individuals or organizations may obtainbiometrics signals from genuine users by intercepting them from thenetwork or obtaining the signals from other applications where the useruses her/his biometrics. The recorded signals can then be reused forunknown, fraudulent purposes such as to impersonate a genuine,registered user of an Internet service. The simplest method is that asignal is acquired once and reused several times. Perturbations can beadded to this previously acquired signal to generate a biometrics signalthat may be perceived at the authentication side as a “fresh” livesignal. If the complete fingerprint or palm print is known to theperpetrator, a more sophisticated method would be to fabricate from, forexample, materials like silicone or latex, an artificial (“spoof”)three-dimensional copy of the finger or palm. Finger- and palm-printimages of genuine users can then be produced by impostors without mucheffort. A transaction server, an authentication server or some othercomputing device would then have the burden of ensuring that thebiometric signal transmitted from a client is a current and live signal,and not a previously acquired or otherwise constructed or obtainedsignal. Using such artificial body parts, many fingerprint andpalm-print readers produce images that look very authentic to a layperson when the right material is used to fabricate these body parts.The images will, in many cases, also appear real to the component imageprocessing parts of the authentication systems. Hence, it is verydifficult to determine whether the static fingerprint or palm-printimages are produced by a real finger or palm or by spoof copies.

PROBLEMS WITH THE PRIOR ART

[0009] Fingerprints and, to a lesser extent, palm prints are used moreand more for authenticating a user's identity for access to medicaldossiers, ATM access, and other such applications. A problem with thisprior art method of identification is that it is possible to fabricatethree-dimensional spoof fingerprints or palm prints. Silicone, latex,urethane or other materials can be used to fabricate these artificialbody parts and many image acquisition devices simply produce a realisticlooking impression of the ridges on the artificial body parts which ishard to distinguish from a real impression. A contributing factor isthat a fingerprint or palm-print impression obtained is the staticdepiction of the print at some given instant in time. The fingerprint innot a function of time. A problem here is that static two-dimensional orthree-dimensional (electronic) spoof copies of the biometrics can befabricated and used to spoof biometric security systems since thesebiometrics are not functions of time.

[0010] Another problem with the prior art is that only one staticfingerprint or palm-print image is grabbed during acquisition of thebiometrcs signal. This instant image may be a distorted depiction of theridges and valley structure on the finger or palm because the userexerts force, torque and/or pressure with the finger with respect to theimage acquisition device (fingerprint or palm-print reader). A problemis that, without grabbing more than one image or modifying the mechanicsof the sensor, it cannot be detected whether the image is acquiredwithout distortion. An additional problem with the prior art of grabbinga static fingerprint image and representing a static fingerprint imageis that there is only one choice for the image that can be used forperson authentication and that image may not be the best depiction ofthe ridge-valley structure.

[0011] The following reference is incorporated by reference in itsentirety:

[0012] Allen Pu and Demetri Psaltis, User identification throughsequential input of fingerprints, U.S. Pat. No. 5,933,515, August 1999.

[0013] The method presented by Pu and Psaltis in their patent U.S. Pat.No. 5,933,515 uses multiple fingers in a sequence which the userremembers and known to the user only. If the fingers are indexed, say,from left to right as finger 0 through finger 9, the sequence is nothingmore than a PIN. If one would consider the sequence plus the fingerprintimages as a single biometric, the sequence is a changeable andnon-static part of the biometric, it is not a dynamic part, because itis not relating to physical force or energy. A problem is that anyonecan watch the fingerprint sequence, probably easier than observing PINentry because fingerprint entry is a slower process. Moreover, itrequires storing each of the fingerprint template of the subject forcomparison.

[0014] Another problem with the prior art is that in order to assureauthenticity of the biometrics signal, the sensor (fingerprint orpalm-print reader) needs to have embedded computational resourcesfinger/palm authentication and sensor authentication. Body partauthentication is commonly achieved by pulse and body temperaturemeasurement. Sensor authentication is achieved with two-directionalchallenge-response communication between the sensor and theauthentication server.

[0015] A potential big problem with prior art palm- and fingerprints isthat if the user somehow loses a fingerprint or palm print impression orthe template representing the print and this ends up in the wrong hands,the print is compromised forever since one cannot change prints. Printsof other fingers can then be used but that can only be done a few times.

[0016] A problem with prior art systems that use static fingerprints isthat there is no additional information associated with the fingerprintwhich can be used for its additional discriminating. That is,individuals that have fingerprints that are close in appearance can beconfused because the fingerprints are static.

[0017] Traditional fingerprint databases may be searched by firstfiltering on fingerprint type (loop, whorl, . . . ). A problem with thisprior art is that there are few fingerprint classes because thefingerprint images are static snapshots in time and no additionalinformation is associated with the fingerprints.

[0018] A final problem with any of the prior art biometrics is that theyare not backward compatible with other biometrics. For example, the useof, say, faces for authentication is not backward compatible withfingerprint databases.

OBJECTS OF THE INVENTION

[0019] An object of the invention is a method for detection ofdistortion in a fingerprint or palm-print image sequence where thesubject moved the finger/palm during acquisition of the sequence.

[0020] Another object of the invention is a method for characterizingdistortion in a fingerprint or palm-print image sequence where thesubject moved the finger/palm during acquisition.

[0021] Another object of the invention is a method for determiningundistorted fingerprint or palm-print images from within the sequence ofprint images.

[0022] A further object of the invention is a system and method forexpressing the rotation of the finger from the distortion of a sequenceof fingerprint images.

[0023] A further object of the invention is a system and method forexpressing the rotation of the palm from the distortion of a sequence ofpalm-print images.

[0024] A further object of the invention is a system and method forexpressing the translation of the palm from the distortion of a sequenceof palm-print images.

[0025] An object of this invention is a resultant fingerprint, acombination of a traditional fingerprint biometric with user-selectedbehavioral changes in the form of rotating the finger. When theresultant fingerprint is compromised, the user can simply select a newbehavioral, dynamic rotational part.

[0026] An object of this invention is a resultant palm-print, acombination of a traditional palm-print biometric with user-selectedbehavioral changes in the form of rotating the palm. When the resultantpalm-print is compromised, the user can simply select a new behavioral,dynamic rotational part.

SUMMARY OF THE INVENTION

[0027] The present invention achieves these and other objectives byextracting the motion (including rotational) component from a sequenceof distorted finger or palm-print images acquired in a time-continuousfashion. The sequence of print images is a resultant fingerprint orpalm-print sequence, a more or less unique characteristic associatedwith a person. A more compact resultant print is a representation of theprint plus a representation of the motion (rotation) as a function oftime.

[0028] There exist physiological and behavioral biometrics.Physiological biometrics are personal characteristics that do notchange, or change very little, over time while behavioral biometrics arecharacteristics which may change over time, and may change abruptly.These behavioral biometrics include user-selected biometrics, like aperson's signature. For the resultant fingerprint and palm-print, thefingerprint or palm-print are physiological biometrics, theuser-selected change, the motion component of the distortion is abehavioral biometric. In a preferred embodiment, a subject produces asequence of fingerprint or palm-print images by rotating the finger orpalm around the vertical axis which modifies the appearance of the printimages using a physical (behavioral) element. The rotation of the fingeris reconstructed from the sequences of the distorted print images, anundistorted print image is selected from the sequence at a point wherethe rotation is zero.

BRIEF DESCRIPTION OF THE DRAWINGS

[0029]FIG. 1 gives prior art examples of biometrics.

[0030]FIG. 2 shows a block diagram of an automated biometrics system forauthentication (FIG. 2A) and a block diagram of an automated biometricssystem for identification (FIG. 2B).

[0031]FIG. 3 shows various components for combining two biometrics atthe system level, where FIG. 3A is combining through ANDing component,FIG. 3B is combining through ORing component, FIG. 3B is combiningthrough ADDing and FIG. 3D is sequential combining.

[0032]FIG. 4 is a generic block diagram conceptually showing thecombining one biometrics with user action (another biometric) to obtaina resultant biometric.

[0033]FIG. 5 is an example of a resultant fingerprint biometrics wherethe user can rotate the finger on the scanner according to auser-defined pattern.

[0034]FIG. 6 is an example of a resultant palm-print biometrics wherethe user can rotate the palm on the scanner according to a user-definedpattern.

[0035]FIG. 7 shows a block diagram of the behavioral componentdetermination of a resultant fingerprint as in FIG. 5 or resultantpalm-print as in FIG. 6.

[0036]FIG. 8 shows the local flow computation on a block by block basisfrom the input resultant fingerprint image sequence.

[0037]FIG. 9 shows the zero-flow and the nonzero-flow blocks of an imageof the sequence.

[0038]FIG. 10 is a plot of the ratio of zero-flow to nonzero-flow blocksas a function of time.

[0039]FIG. 11 explains the computation of the curl or the spin of thefinger as a function of time, which is the behavioral component of theresultant fingerprint.

[0040]FIG. 12 shows the radial sampling process around the bounding boxof a stationary part in a print image.

[0041]FIG. 13 is a flow diagram of the thresholding classificationoperation to detect distortion in a print sequence.

DETAILED DESCRIPTION OF THE INVENTION

[0042] This invention introduces a new method to process a new biometriccalled resultant biometrics. A resultant biometrics is a sequence ofconsecutive physiological or behavioral biometrics signals recorded atsome sample rate producing the first biometrics signal plus a secondbiometrics, the behavioral biometrics, which is the way thephysiological or behavioral biometrics is transformed over some timeinterval. This transformation is the result of a series ofuser-controlled changes to the first biometric.

[0043] Traditional biometrics, such as fingerprints, have been used for(automatic) authentication and identification purposes for severaldecades. Signatures have been accepted as a legally binding proof ofidentity and automated signature authentication/verification methodshave been available for at least 20 years. FIG. 1 gives examples ofthese biometrics. On the top-left, a signature 110 is shown and on thetop-right a fingerprint impression 130 is shown.

[0044] Biometrics can be used for automatic authentication oridentification of a subject. Typically, the subject is enrolled byoffering a sample biometric when opening, say, a bank account orsubscribing to an Internet service. From this sample biometrics, atemplate is derived that is stored and used for matching purposes at thetime the user wishes to access the account or service. In the presentpreferred embodiment, a template for a resultant biometric is acombination of a traditional template of the biometrics and a templatedescribing the changing appearance of the biometric over time.

[0045] Resultant fingerprints and palm prints are described in furtherdetail. A finger- or palm print template is derived from a selectedimpression in the sequence where there is no force, torque or rollingexerted. The template of the trajectory is a quantitative description ofthis motion trajectory over the period of time of the resultantfingerprint. Matching of two templates, in turn, is a combination oftraditional matching of fingerprint templates plus resultant stringmatching of the trajectories similar to signature matching. Resultantfingerprints sensed while the user only exerts torque are described ingreater detail.

[0046] A biometric more or less uniquely determines a person's identity,that is, given a biometric signal, the signal is either associated withone unique person or narrows down significantly the list of people withwhom this biometric is associated. Fingerprints are an excellentbiometrics, since never in history two people with the same fingerprintshave been found; on the other hand, biometrics signals such as shoe sizeand weight are poor biometrics signals since these signals have littlediscriminatory value. Biometrics can be divided up into behavioralbiometrics and physiological biometrics. Behavioral biometrics depend ona person's physical and mental state and are subject to change, possiblyrapid change, over time. Behavioral biometrics include signatures 110and voice prints 120 (see FIG. 1). Physiological biometrics, on theother hand, are subject to much less variability. For a fingerprint, thebasic flow structure of ridges and valleys, see the fingerprint 130 inFIG. 1, is essentially unchanged over a person's life span. As anexample of another biometrics, the circular texture of a subject's iris,140 in FIG. 1, is believed to be even less variable over a subject'slife span. Hence, there exist behavioral biometrics, e.g., 110 and 120,which to a certain extent are under the control of the subjects andthere exist physiological biometrics whose appearance cannot beinfluenced (the iris 140) or can be influenced very little (thefingerprint 130). The signature and voice print on the left arebehavioral biometrics; the fingerprint and iris image on the right arephysiological biometrics.

[0047] Referring now to FIG. 2A. A typical, legacy automatic fingerprintauthentication system has a fingerprint image (biometrics signal) asinput 210 to the biometrics matching system. This system consists ofthree other stages 215, 220 and 225, comprising: signal processing 215for feature extraction, template extraction 220 from the features andtemplate matching 225. Along with the biometrics signal 210, anidentifier 212 of the subject is an input to the matching system. Duringthe template matching stage 225, the template associated with thisparticular identifier is retrieved from some database of templates 230indexed by identities. If there is a Match/No Match between theextracted template 220 and the retrieved template from database 230, a‘Yes/No’ 240 answer is the output of the matching system. Matching istypically based on a similarity measure, if the measure is significantlylarge, the answer is ‘Yes,’ otherwise the answer is ‘No.’ The followingreference describes examples of the state of the prior art:

[0048] N. K. Ratha, S. Chen and A. K. Jain, Adaptive flow orientationbased feature extraction in fingerprint images, Pattern Recognition,vol. 28, no. 11, pp. 1657-1672, November 1995. This reference isincorporated herein by reference in its entirety.

[0049] Note that system 200 is not limited to fingerprintauthentication, this system architecture is valid for any biometric. Thebiometric signal 210 that is input to the system can be acquired eitherlocal to the application on the client or remotely with the matchingapplication running on some server. Hence architecture 200 applies toall biometrics and networked or non-networked applications.

[0050] System 200 in FIG. 2A is an authentication system, system 250 inFIG. 2B is an identification system. A typical, legacy automaticbiometrics signal identification system takes only a biometric signal210 as input (FIG. 2A). Again, the system consists of three other stages215, 220 and 225, comprising: signal processing 215 for featureextraction, template extraction 220 from the features and templatematching 225. However, in the case of an identification system 250, onlya biometric signal 210 is input to the system. During the templatematching stage 225, the extracted template is matched to all template,identifier pairs stored in database 230. If there exists a match betweenthe extracted template 220 and a template associated with an identity indatabase 230, this identity is the output 255 of the identificationsystem 250. If no match can be found in database 230, the outputidentity 255 could be set to NIL. Again, the biometric signal 210 can beacquired either local to the application on the client or remotely withthe matching application running on some server. Hence architecture 250applies to networked or non-networked applications.

[0051] Biometric signals can be combined (integrated) at the systemlevel and at the subject level. The latter is the object of thisinvention. The former is summarized in FIG. 3 for the purposes ofcomparing the different methods and for designing decision methods forintegrated subject-level biometrics (resultant biometrics). Fourpossibilities for combining (integrating) two biometrics are shown:Combining through ANDing 210 (FIG. 3A), Combining through ORing 220(FIG. 3B), Combining through ADDing 230 (FIG. 3C), and serial orsequential combining 240 (FIG. 3D). Two biometrics B_(x) (250) and B_(y)(260) of a subject Z are used for authentication as shown in FIG. 3.However, more than two biometrics of a subject can be combined in astraightforward fashion. These biometrics can be the same, e.g., twofingerprints, or they can be different biometrics, e.g., fingerprint andsignature. The corresponding matchers for the biometrics B_(x) andB_(y), are matcher A 202 and matcher B 204 in FIG. 3, respectively.These matchers compare the template of the input biometrics 250 and 260with stored templates and either give a ‘Yes/No’ 214 answer as insystems 210 and 220 or score values, S₁ (231) and S₂ (233), as insystems 230 and 240.

[0052] System 210, combining through ANDing, takes the two ‘Yes/No’answers of matcher A 202 and matcher B 204 and combines the resultthrough the AND gate 212. Hence, only if both matchers 202 and 204agree, the ‘Yes/No’ output 216 of system 210 is ‘Yes’ (the biometricsboth match and subject Z is authenticated) otherwise the output 216 is‘No’ (one or both of the biometrics do not match and subject Z isrejected). System 220, combining through ORing, takes the two ‘Yes/No’answers of matchers A 202 and B 204 and combines the result through theOR gate 222. Hence, if one of the matchers' 202 and 204 ‘Yes/No’ output216 is ‘Yes,’ the ‘Yes/No’ output 216 of system 220 is ‘Yes’ (one orboth of the biometrics match and subject Z is authenticated). Only ifboth ‘Yes/No’ outputs 214 of the matchers 202 and 204 are ‘No,’ the‘Yes/No’ output 216 of system 220 is ‘No’ (both biometrics do not matchand subject Z is rejected).

[0053] For system 230, combining through ADDing, matcher A 202 andmatcher B 204 produce matching scores S₁ (231) and S₂ (233),respectively. Score S₁ expresses how similar the template extracted frombiometrics B_(x) (250) is to the template stored in matcher A 202, whilescore S₂ expresses how similar the template extracted from biometricsB_(y) (260) is to the template stored in matcher B 204. The ADDer 232gives as output the sum of the scores 231 and 233, S₁+S₂. In 234, thissum is compared to a decision threshold T, if S₁+S₂>T 236, the output is‘Yes’ and the subject Z with biometrics B_(x) and B_(y) isauthenticated, otherwise the output is ‘No’ and the subject is rejected.

[0054] System 240 in FIG. 3 combines the biometrics B_(x) (250) andB_(y) (260) of a subject Z sequentially. First biometrics B_(x) (250) ismatched against the template stored in matcher A (202) resulting inmatching score S₁ (231). The resulting matching score is compared tothreshold T₁ 244, and when test 244 fails and the output 238 is ‘No’ thesubject Z is rejected. Otherwise biometrics B_(y) (260) is matchedagainst the template stored in matcher B (204). The output score S₂(233) of this matcher is compared to threshold T₂ 246. If the output is‘Yes,’ i.e., S₂>T₂ (236) subject Z is authenticated. Otherwise, when theoutput is ‘No’ 238, subject Z is rejected.

[0055]FIG. 4 is a generic block diagram for combining a biometrics withuser action, i.e., combining biometrics at the subject level. The useraction, just like the movement of a pen to produce a signature, is thesecond behavioral biometrics. The user 410 offers a traditionalbiometric 420 for authentication or identification purposes. Such abiometrics could be a fingerprint, iris or face. However, rather thanholding the biometrics still, as in the case of fingerprints or faces,or keeping the eyes open, as in case of iris recognition, the userperforms some specific action 430, a(t) with the biometrics. This actionis performed over time 432, from time 0 (434) to some time T (436).Hence, the action a(t) is some one-dimensional function of time 430 andacts upon the traditional biometric 420. Note that this biometric is theactual biometric of user 410 and not a biometrics signal (i.e., in thecase of fingerprints, it is the three-dimensional finger with the printon it). It is specified what the constraints of the action 430 are butwithin these constraints, the user 410 can define the action. (Forexample, constraints for putting a signature are that the user can movethe pen over the paper in the x- and y-direction but cannot move the penin the z-direction.) That is, the action 430 in some sense transformsthe biometric of the user over time. It is this transformed biometric450 that is input to the biometric signal recording device 460. Theoutput 470 of this device is a sequence of individually transformedbiometrics signals B(t) 480 from time 0 (434) to some time T (436). Inthe case of fingerprints, these are fingerprint images, in the case offace, these are face images. This output sequence 470, is the input 485to some extraction algorithm 490. The extraction algorithm computes fromthe sequence of transformed biometrics the pair (a′(t), B), 495, whichis itself a biometric. The function a′(t) is some behavioral way oftransforming biometric B over a time interval [0, T] and is related tothe function a(t) wich is chosen by the user (very much like a userwould select a signature). The biometrics B can be computed from thepair (a′(t), B), that is, where a(t) 430 is zero, where there is noaction of the user, the output 470 is undistorted copy of biometrics420. In general, it can be computed where in the signal 480, thebiometrics 420 is not distorted.

[0056] Refer to FIG. 5. This figure is an example of a resultantfingerprint biometric where the user can rotate the finger on thefingerprint reader 510 (without sliding over the glass platen). Thisrotation can be performed according to some user defined angle a as afunction of time a(t). An example of producing a resultant fingerprintis given in FIG. 5. The user puts the finger 540 on the fingerprintreader 510 in hand position 520. Then from time 0 (434) to time T (436),the user rotates finger 540 over the glass platen of fingerprint reader510 according to some angle a as a function of time a(t). The rotationtakes place in the horizontal plane, the plane parallel to the glassplaten of the fingerprint reader. The rotation function in this case isthe behavioral part of the resultant fingerprint and is defined by theuser. (If this portion of the resultant biometric is compromised, theuser can redefine this behavioral part of the resultant fingerprint.)First the user rotates by angle 550 to the left, to the hand position525. Then the user rotates by angle 555 to the right, resulting in finalhand position 530. During this operation over time interval [0, T], thefingerprint reader has as output 470 a sequence of transformed(distorted) fingerprint images. This output 470 is a sequence oftransformed biometrics 480 (fingerprints), as in FIG. 4, which are theinput to the extraction algorithm 490 (FIG. 4). This algorithm computes,given the output 470, the angle a as a function of time a(t) 560 overthe time interval 0 (434) to time T (436). The resultant fingerprint inthis case is (a(t), F), with F the undistorted fingerprint image. Theundistorted fingerprint image is found at times 434, 570 and 436 wherethe rotation angle a is zero. A method for extracting the rotationangles from the distorted fingerprint images is described in FIGS. 9-11.

[0057]FIG. 6 gives an example of the same principle as fingerprints forpalm prints. The palm print reader 610 with glass platen 620 can, forexample, be mounted next to a door. Only authorized users with matchingpalm print templates will be allowed to access. The user will puthis/her hand 630 on the palm print reader platen 620. As with theresultant fingerprints of FIG. 5, the user will not keep the palmbiometrics still but rather rotate the palm. In the case of FIG. 6,rotation of the palm around the axis perpendicular to the glass platenis the behavioral part of the resultant palm-print biometric. The usercould, for instance, rotate the hand to the right 640, followed by arotation of the hand to the left 644, followed by a rotation of the handto the right 648 again. As in FIG. 5, during these operations over sometime interval [0, T], the palm print reader has as output a sequence oftransformed (distorted) palm print images. This output is a sequence oftransformed biometrics 480 (palm prints), as in FIG. 4, which are theinput to an extraction algorithm 490 as in FIG. 4. The algorithmcomputes, given the output of palm print reader 610, the palm rotationangle a as a function of time a(t) 560 over the time interval 0 (434) totime T (436). The resultant palm print in this case will be (a(t), P),with P the undistorted palm print image. The undistorted palm printimage is found at times 434, 570 and 436 where the rotation angle a iszero.

[0058] In FIG. 7, a block diagram 700 of a generic process forextracting the behavioral component from a resultant fingerprint orpalm-print is given. The input 710 is a sequence of print images B(t).In block 720, two subsequent biometric images, B(t+1) and B(t), areprocessed through inter-signal analysis. Block 730, uses this analysisto extract the change, a(t+1)−a(t), in the rotational component. Inturn, this gives the output a(t) as a function of time 740, where a(t)is the rotation of the resultant finger or palm-print B(t). Added inFIG. 9 are the specific steps for estimating the finger rotation from asequence of distorted fingerprint images produced as in FIG. 5 or forestimating the palm rotation from a sequence of distorted palm-printimages produced as in FIG. 6. Step 720 amounts to inter-image flowanalysis, determining the motion of the image pattern from image toimage for each block. Step 730 amounts to determining the overall motionof all the image blocks in the form of an affine transformation, atransformation specified by transformation, rotation and shear. Theseare further explained in FIGS. 8-11.

[0059] Rotation from one print image to the next can be estimated usingthe following steps illustrated in FIG. 8. The images, B(t) and B(t+1),810 and 815, are divided up into 16×16 blocks 820, 822, 824, . . . , 828as determined by the MPEG compression standard. Given a fingerprintimage sequence B(t), of which two images are shown in FIG. 8, theinter-image flow (u, v) 840 for each block (of size 16×16) 830 presentin an image is computed. This represents the motion that may be presentin any image B(t) 810 with respect to its immediate next image B(t+1)815 in the sequence. A flow characterization [u(x,y), v(x,y)] 850 as afunction of (x, y) 860 and t 870 of an image sequence is then a uniformimage-to-image motion representation amenable for consistentinterpretation. This flow 850 for the 16×16 blocks in each image can becomputed from the raw motion vectors encoded in the MPEG-1 or MPEG-2image sequences. If the input is uncompressed, the flow field can beestimated using motion estimation techniques known in the prior art.

[0060] The following references describe the state of the prior art inMPEG compression, an example of prior art optical flow estimation inimage sequences, and an example of prior art of directly extracting flowfrom MPEG-compressed image sequences respectively:

[0061] B. G. Haskell, A. Puri and A. N. Netravali, Digital Video: Anintroduction to MPEG-2, Chapman and Hill, 1997.

[0062] J. Bergen, P. Anandan, K. Hanna and R. Hingorani, Hierarchicalmodel-based motion estimation, Second European Conference on ComputerVision, pp. 237-252, 1992.

[0063] Chitra Dorai and Vikrant Kobla, Extracting Motion Annotationsfrom MPEG-2 Compressed Video for HDTV Content Management Applications,IEEE International Conference on Multimedia Computing and Systems, pp.673-678, 1999.

[0064] The process described in FIG. 8 gives the flow of each 16×16block in each frame in the given sequence of print images. Next, usingthis flow, those images are selected that exhibit a high level ofinter-image flow activity. Referring to FIG. 9, for each individualimage, the image is scanned from left-to-right and top-to-bottom. Thenumber Z of zero-flow blocks 820, 822, 824, . . . , 930, 932, . . . ,934, . . . , 828 and the number NZ of non-zero flow blocks 920, 922,924, . . . , 928 are determined. The ratio Z/NZ then gives aquantitative characterization of the total flow present in the image.During distortion, while a portion of the finger is held stationary, therest of the finger is twisted, rolled or pressed hard on the scanningsurface and this results in a flow field in which there are a fewzero-flow blocks corresponding to the stationary part of the finger anda substantial number of surrounding blocks show some non-zero flow. IfZ/NZ<1, the number of blocks with non-zero flow exceeds the number ofblocks with zero flow. Plotting this ratio as a function of time resultsin a graph as shown in FIG. 10. The horizontal axis 1020 represents timet 1040 and the vertical axis 1010 represents the ratio Z/NZ 1030. Theplot 1050 then indicates time intervals 1070 and 1080 where there existssignificant flow from image to image.

[0065] For images with Z/NZ>>1, there exist many more blocks withoutimage flow than block with image flow. These images can be used toobtain finger or palm-print images with minimal distortion. If Z/NZ<1 inan image, the image is deemed to be a candidate for detailed distortionanalysis.

[0066] When the palm or finger is rotated during acquisition, a portionof the finger is typically held stationary and the rest of the printaround this pivotal unmoving region is moved to introduce distortion inthe print image. In FIG. 9, the pivotal region 940 with blocks 930, 932,. . . , 934 is surrounded by an area 950 of moving blocks 920, 922, 924,. . . , 928. In images where the number of blocks with non-zero flowexceeds the number of blocks with zero flow (referred to as candidateimages), connected components of zero-flow blocks are found. (Theprocess of finding connected image components is well-known to thoseskilled in the art.) In each candidate image, the largest region,measured in area, of zero-flow blocks 940 is selected as the unmovingregion which is used in the process to determine rotation of the print.

[0067] Refer now to FIG. 11. By examining the flow [u(x,y), v(x,y)] 850in the blocks 920, 922, 924, . . . , 928 of FIG. 9, a largest connectedcomponent of zero-motion blocks, pictured by pivotal region 1110 in FIG.11 (940 in FIG. 9) is determined in each candidate image. The rotationestimation process is performed on the flow around this region. Usingthe flow computed for each image and each block in the given imagesequence, motion parameters from the fingerprint region are computed bydetermining an affine motion model for the consecutive image-to-imageflow and sampling the non-zero motion blocks radially around thebounding box 1120 of region 1110. FIG. 12 shows the radial samplingprocess in detail. Around the bounding box 1120 of the pivotal region1110, annular regions of moving blocks 1220, 1230 and so on, areselected.

[0068] Affine motion M 1130 can transform shape 1140 into shape 1145 inFIG. 11B and quantifies translation 1150, rotation 1152 and shear 1154due to image flow. Six parameters, a₁ . . . a₆ are estimated in thisprocess, where a₁ and a₄ correspond to translation, a₃ and a₅ correspondto rotation, and a₂ and a₆ correspond to shear. This transformation(motion) is given by

u(x,y)=a ₁ +a ₂ x+a ₃ y

u(x,y)=a ₄ +a ₅ x+a ₆ y

[0069] The affine transformation parameters are estimated using all themotion blocks in all annular regions around bounding box 1120 by usingthe least square error estimation technique described in

[0070] J. Meng and S. -F. Chang, “CVEPS—a compressed video editing andparsing system,” in Proc. ACM 1996 Multimedia Conference, November.1996. This reference is incorporated herein by reference in itsentirety.

[0071] Average curl is computed in each image t, as C(t)=−a₃+a₅. Thecurl in each image quantitatively provides the extent of rotation, orthe spin of the finger or palm skin around the pivotal region. That is,an expression C(t) of the behavioral component of the resultantfingerprint or palm-print computed from flow vectors [u(x,y), v(x,y)]850 is obtained. The magnitude of the average translation vector,T(t)=(a₁, a₄) of the frame is also computed.

[0072] A smoothed curl (rotation) C′(t) 1310 in FIG. 13 and translationT′(t) as a functions of time are determined by the computing the averagevalues of curl and translation over a temporal neighborhood. Thepreferred temporal window is one-tenth of a second, or in other wordsthree consecutive images. Hence, C′(t) is the behavioral rotationcomponent of the resultant palm-print or fingerprint.

[0073] To detect where distortion (for example, rotation) occurs in theprint sequence and to select undistorted images in a sequence of printimages, a simple thresholding-based classification operation is carriedout on the sequences of smoothed average values of curl 1310 in FIG. 13and translation as functions of time for the print sequence. (Theprocess of thresholding is well-known to those skilled in the art.) Thisoperation makes use of the temporal length of the sequence of contiguouscandidate images and the range of their curl values as a function oftime in its final determination of occurrence of distortion in thesequence. If the temporal length, T of a group of contiguous candidateimages 1320 in FIG. 13 exceeds a threshold, t_b 1330 in FIG. 13 (t_b=0.1seconds in the best embodiment), then it is established that the groupcannot be a noisy blip but rather it contains frames that have low Z/NZratio and have significant smoothed curl values. Therefore that group1320 in FIG. 13 is labeled to contain distorted images 1340 in FIG. 13.On the other hand, if the temporal length, T of a group of contiguouscandidate images 1320 in FIG. 13 is small (less than t_b), then theclassification process investigates more closely to verify whether thissmall blip is a true distortion event. This is carried out using twosequential tests: (I) the first test 1350 in FIG. 13 checks whether therange of curl values of images in this group exceeds a certain thresholdC_v; this establishes that there are abrupt and large changes in thecurl values which are indicators of distortion due to quick fingertwist. (II) the second test 1360 in FIG. 13 examines whether the maximumtranslation magnitude in this group of frames 1320 in FIG. 13 is lessthan a certain threshold T_v; this is to ensure that the group of framesdoes not undergo pure translation which can only be due to a fingerbeing merely moved around from one point on the scanning surface toanother without any twist. Once the group is shown to possess high curlvariation and low translation, the classification process labels thegroup as a true distortion group 1340 in FIG. 13. If the curl variationis low, it is inferred that there is no strong evidence of distortionand the group is marked as an undistorted interval 1370 in FIG. 13.Affine parameters of the selected distorted images 1340 in FIG. 13characterize the finger movements in these images leading to distortion.At the end of the thresholding-based classification operation, eachimage in the print sequence has one of two labels, distortion andno-distortion. A merging process is then carried out to groupconsecutive time intervals of identical distortion or no-distortionlabels to form cohesive distorted or undistorted sub sequences in theprint sequence respectively. Images can be selected from the undistortedsub sequences within the sequence of print images to match with storedtemplates.

[0074] For the resultant prints discussed, we have a traditionalbehavioral or physiological biometric. For representation (template)purposes and for matching purposes of that part of resultant biometrics,these traditional biometrics are well understood in the prior art. (See,the above Ratha, Chen and Jain reference for fingerprints.) For theother part of the resultant prints, the behavioral part, we are leftwith some one-dimensional rotation C′(t) of time, the user action.Matching this part amounts to matching this function C′(t) with a storedtemplate S(t). Such matching is again well-understood in the prior artand is routinely done in the area of signature verification. Thefollowing reference gives examples of approaches for matching.

[0075] V. S. Nalwa, “Automatic on-line signature verification,”Proceedings of IEEE, pp. 215-239, February 1997. This reference isincorporated herein by reference in its entirety.

[0076] Now the resultant print, after matching with a stored templatehas either two ‘Yes/No’ (214 in FIG. 3) answers or two scores S₁ and S₂(231 and 233 in FIG. 3). Any of the methods for combining the twobiometrics discussed in FIG. 3 can be used to combine the traditionaland user-defined biometrics of a resultant biometric to arrive at amatching decision.

We claim:
 1. A biometrics system comprising: an acquisition device foracquiring and storing a sequence of discrete print images from a part ofa hand moving during a time period; a trajectory process that determinesthe position and orientation of the images of said part of the hand as afunction of time during the time period; and an estimator process thatdetermines a distortion of the discrete print images as a function oftime due to the change in position and orientation, wherein theestimator process determines distortion by determining at least a motionof an image pattern occurring in one or more blocks of at least two ofthe discrete print images.
 2. A system, as in claim 1, where the part ofthe hand includes one or more of the following: a fingerprint and apalm-print.
 3. A system, as in claim 1, where the distortion is causedby one or more of the following: rotation, translation, and shear.
 4. Asystem, as in claim 1, where the motion is interframe motion and wherethe estimator process comprises the steps of: determining one or moreblocks of interframe motion between consecutive pairs of the images inthe sequence; determining a proportion of blocks with no motion toblocks with some motion; using the proportion to select a set ofcandidate distorted images; identifying a largest stationary andspatially contiguous block in each candidate distorted image in the set;estimating a global affine transformation between every pair ofcandidate distorted images in the set about the stationary andcontiguous block; determining a curl and translation from the globalaffine transformation between every pair of candidate distorted imagesin the set; and using the change of the curl over the time period toindicate the distortion.
 5. A system, as in claim 4, where, when thechange in curl over the time period is greater than a threshold, thedistortion is caused by one or more of the following: rotation,translation, and shear.
 6. A system, as in claim 4, where the distortionis primarily translation when curl is within a second threshold of zeroand the translation exceeds a third threshold.
 7. A method for detectingthe distortion of a fingerprint or palm-print, comprising the steps of:acquiring and storing a sequence of discrete print images from a part ofa hand moving during a time period; determining the position andorientation of the images of said part of the hand as a function of timeduring the time period; and determining a distortion of the discreteprint images as a function of time due to the change in position andorientation, wherein the step of determining a distortion furthercomprises the step of determining at least a motion of an image patternoccurring in one or more blocks of at least two of the discrete printimages.
 8. The method of claim 7, where the part of the hand includesone or more of the following: a fingerprint and a palm-print.
 9. Themethod of claim 7, where the distortion is caused by one or more of thefollowing: rotation, translation, and shear.
 10. A method fordetermining a distortion of a set of images as a function of time due tothe change in position and orientation of a hand comprising the stepsof: determining one or more blocks of interframe motion betweenconsecutive pairs of images in a sequence of images from a part of ahand moving during a time period; determining a proportion of blockswith no motion to blocks with some motion; using the proportion toselect a set of candidate distorted images; identifying a largeststationary and spatially contiguous block in each candidate distortedimage in the set; estimating a global affine transformation betweenevery pair of candidate distorted images in the set about the stationaryand contiguous block; determining a curl and translation from the globalaffine transformation between every pair of candidate distorted imagesin the set; and using the change of the curl over the time period toindicate the distortion.
 11. The method of claim 10, where, when thechange in curl over the time period is greater than a threshold, thedistortion is caused by one or more of the following: pure rotation,translation, and shear.
 12. The method of claim 10, where the distortionis primarily translation when curl is within a second threshold of zeroand the translation exceeds a third threshold.
 13. A biometrics systemcomprising: an acquisition device for acquiring and storing a sequenceof discrete images from a part of a hand moving during a time period; atrajectory process that determines the position and orientation of theimages of said part of the hand as a function of time during the timeperiod; an estimator process that determines a distortion of thediscrete images as a function of time due to the change in position andorientation, wherein the estimator process determines distortion bydetermining at least a motion of an image pattern occurring in one ormore blocks of at least two of the discrete print images; andidentifying a person utilizing at least said determined distortion. 14.A biometrics system comprising: an acquisition device for acquiring andstoring a sequence of discrete images from a part of a hand movingduring a time period; a trajectory process that determines the positionand orientation of the images of said part of the hand as a function oftime during the time period; an estimator process that determines adistortion of the discrete images as a function of time due to thechange in position and orientation, wherein the estimator processdetermines distortion by determining at least a motion of an imagepattern occurring in one or more blocks of at least two of the discreteprint images; and authenticating a person utilizing at least saiddetermined distortion.